MacFormat 1994 November/macformat-018.iso/Comms Spectacular/mosaic/Helpers/Sparkle202 Folder.1/Docs/README 2nd

Text File | 1994-04-09 | 28.7 KB | 559 lines | [TEXT/ALFA]

Sparkle: A new mac MPEG player.
-------------------------------
Documents for version 1.7
This document looks best in 9 point monaco.
On an MPW marks aware editor (like Alpha) this document has marks for
each section.

This document contains the following sections:
INTRODUCTION
LEGALITIES
SAVING TO MPEG
FAQs
WHY DOESN'T SPARKLE DO....
SMALL THINGS YOU MIGHT NOT HAVE NOTICED
THE FUTURE
HOW CAN YOU HELP IMPROVE SPARKLE?

------------------------------------------------------------------------------
INTRODUCTION

Hi there, friendly users. This is release 2.0 of my mac MPEG player.
Version 1.0 of this code was based on the Berkeley MPEG unix code.
(Anyone who wants to play with the Berkeley code can get it from
toe.cs.berkeley.edu in pub/multimedia/mpeg.) It was released as soon as
it was usable. This version has had most of the Berkeley code rewritten
for greater speed, smaller memory footprint, and more flexibility.

Version 2.0 of this code makes use of the Berkeley MPEG encoder, available
from the same place as above. Like version 1.0, this Berkeley code has not
been touched much except to make it Mac aware, object oriented, and thread
based. As a result, MPEG encoding is slow and a ridiculous memory hog
(what do you expect from UNIX coders---they seem to have no interest in
elegant design)?
As time permits I'll be going through that encoding code to make it faster
and use less memory.

Notice that there is a section of this document called FAQs.
Please read it before sending me mail, thus avoiding wasting your time and
mine.

Please read all of this document before playing with the program. While
actually using the program is pretty simple, there are a few things you
should be aware of. Many of you may not care when I waffle on about
technical details. However I would ask all programmer readers (especially
people knowledgable about QuickTime, and especially Apple [and
ex-Apple :-( employees] to look at the tech sections and help me out with
comments and suggestions. Each time someone gives me a pointer on how to
do something it cuts a week or more off the release date of the next
version of Sparkle.

Features:
•    Standard mac interface with menus and windows.
•    Uses the QuickTime movie controller to control the MPEG viewing.
•    MultiFinder friendly, with good backgrounding behavior.
•    Saves MPEGs to QuickTime movies.
•    Can open multiple files at once.
•    Free.

To run it needs at least:
•    System 7.
•    QuickTime 1.6
•    A 68020 or better.
•    800K to play one 160x120 MPEG---to open more or large MPEGs increase the
    partition.
    Encoding MPEGs requires much more memory---about a minimum of 1600K for a
    160x120 source.
•    It does not require the Thread Manager to be installed, but runs much
    better if it's present. Right now, the Thread Manager is an extension
    that you can get from any friend who knows a lot about macs, or from the
    apple ftp site. I wish I could distribute it with Sparkle, but Apple's
    boneheaded lawyers won't allow that.
    (Question for you as a user, does it make your Macintosh expereince
    better or worse to find that a vital piece you need to run a program is
    not available with that program? Now try to explain that to Apple
    management, who sometime seem determined to make using a mac as difficult
    an experienc as using a PC.)
    The Thread Manager may be built into System 7.5 when that's released, at
    which point you won't need the extension.

This program works fine, with good handling of errors, on my Quadra 610, but
that's the only machine I have to test it. If you find a bug that is not
caused by the various things listed below, please mail me with as many
details as possible, both about your machine and about what the  program
was last doing before it died on you.

I don't think there's much to say on the use of this program---you pretty
much run it like any other mac program. There is a section in this
document on tips that may not be obvious.
When opening files, you can choose to show all files, or only files with a
.mpg suffix.
If you choose the "show all files" option and open some random file, don't
be surprised when you are told that that is not a valid MPEG file.
If you set the option to change file types, the file type of the MPEG file
you are changing will be changed to a Sparkle file, which will give it a
nice icon and allow you to open the file by double-clicking on it.

Underneath the movie controller for each window is a progress bar which
will update itself when slow things are happening. If you have a fast
machine (say a Quadra or Centris) this won't have to update itself much,
except if you open large (and not too common) MPEGs---say 320x240. If you
really want to see what the progress update looks like, try having lots of
MPEGs playing at once, then switch Sparkle into the background.
If you save an MPEG to QuickTime using Cinepak, your machine may appear to
freeze while each frame is being converted. I have installed code to
reduce the extent of the freeze, but basically this is a bug with
QuickTime---the system just grabs control and won't let go for about five
seconds. Maybe this'll be fixed in QuickTime 2.0 if we're lucky.

I have tested this program extensively under low-memory conditions when it
opens files and plays them.  It should never crash under those conditions.
In 600K you can easily open, play and save to QT a 120x160 I-frame MPEG.  In
1500K you can open, play and save to QT a 320x240 IBP-frame MPEG.

Disk errors in various forms (bad sectors reading an MPEG file, no disk
space writing an QuickTime file, etc) will not crash, but the system will
put up an error alert and not handle the error very well (for example you
won't be given a chance to destroy old files to free up space on a disk).
Decent recovery from disk errors is on the list of things to do.

------------------------------------------------------------------------------
LEGALITIES

This program can be freely distributed.
If you want to include it on a CD-ROM collection, please ask me first,
but I'll probably allow you.
NOTE: Some authors will not allow their stuff to be distributed on CD-ROMs
for which people have to pay. I would ask these authors to consider things
more carefully. Many people (like myself) do not have very good ftp access
and a CD ROM of stuff is a godsend, even if we do have to pay a little for
it. Think about it.
The only people I won't allow to carry it are ZiffNet because I feel their
download policies are unfair. (As I understand things, ZiffNet put up for
downloading stuff they pull off the InterNet, but won't allow the InterNet
to distribute their stuff. If I am wrong about this, someone please tell
me and I'll change this clause.)

If you feel a desperate need to thank me for this program, send me a
floppy of interesting MPEGs or QT movies you've picked up.
(Please don't e-mail me large files without warning.)
My paper-mail address is
    Maynard Handley
    ADI
    284 Stuart Street
    Dunedin
    New Zealand

------------------------------------------------------------------------------
SAVING TO MPEG

The dialog box offering options when you save to MPEG is not that great
right now if you don't know what you're doing. I'm sorry---I'll fix it
when I have time, like everything else.

For a better explanation of how MPEG works, read the Technical Notes file
in this package, but here are rough details.

When encoding your MPEG you will need to decide what types of frames you
want to use. If you want your MPEG to be read widely, ie on PC systems,
you should consider using only I-frames. The most common MPEG decoder on
PCs is the Xing decoder which only understands the simple I-frame MPEG
format. If you want to get the best possible compression you should use
I, P and B frames, but then your MPEG will only be able to be read by
smarter decoders. A compromise is to use only I and P frames. This will
give you a lot of the compression benefit of I, P and B frames, but will
be faster to encode.

If you don't want to know the details of these frames, choose a frame
pattern from the popup menu.
If you want to create your own frame pattern, here are some details:
(If you don't want to create your own frame pattern, ignore these.)
1) The first frame in the MPEG will always be I. That implicit I frame is
    displayed in front of the frame pattern text you enter because it is
    always there and you don't need to put it in. That implicit I frame is
    only used for the first frame, after that your pattern is repeated as
    necessary. Thus if your pattern is BBPBBI, the encoder will encode frames
    using the sequence I BBPBBI BBPBBI BBPBBI ...
    (For people who care about MPEG details, this is the playback order
    of the frames. The actual order of the frames stored in the file
    will be be I PBBIBB PBBIBB PBBIBB.)
2) If you are using B frames, you have to start off with two referential
    frames, the initial I frame then either an I or a P frame.
3) The last few frames of the MPEG, if they would be B frames, will be
    converted to I-frames. (If this didn't occur, like in the original
    Berkeley encoder code before I munged it, you will lose those final
    frames.)
4) Don't create too long a pattern of Ps or Bs. You'll start to lose
    quality and random access into the movie will become more granular. The
    presets I've given should guide you as to sensible selections.

Once you have chosen a frame pattern, you'll need to select the amount of
compression you want to use. This is set by the I, P and B quantization
levels. The values the dialogs gives as defaults will usually make sense
but if you want to change them:
1) The values are restricted to the range 1 through 1024 inclusive.
    A quantization of 1 is the highest quality, a quantization of 1024 is the
    lowest quality.
2) Normal quality is a quantization of about 8. Low quality is a
    quantization of about 16. High quality is about 6.
3) So why give a range all the way up to 1024?
    If you must know, I foresee some users wanting to see what happens as you
    change this value. You may hope to obtain an MPEG coded using only DC
    components and with all AC as zero. However if you do this with lowish
    quantizations, like 64, you'll have unexpected results---see the Tech
    Notes for why. So really the 1024 limit is a hokey way to generate MPEGs
    with only DC components.

You can choose from various algorithms for generating P and B frames. I
haven't explored these in detail and they may change when I have time to
look at them. For now the important things to note are
1) They all seem to generate about the same quality, and about the same
    file size.
2) They are ordered in the menus by speed, with the fastest ones first.
3) The B-frame exhaustive search will take approx forever to do anything.
    Try it once to see what I mean, but don't expect to use it for anything
    useful.
As I explore these more and start diddling the code, I'll provide more
explanation.

If you save to MPEG, because of the non-causal algorithms used by MPEG
compression, frames are not compressed in the order they are displayed.
This means that if you are using B-frames and stop the compression before
it reaches the end of the source movie, the encoder may have queued up
some frames it needs to compress to make the MPEG file consistent.
If this is the case, Sparkle will have to encode those queued frame before
it stops the compression, so it won't stop immediately.
The message window will tell you how many frames have been queued so you can see
how things are going. If you have the speech manager installed, Sparkle
will also tell you how many frames are left, which you amy find more
convenient.

------------------------------------------------------------------------------
FAQs (Frequently asked questions).

• Why when I try to step back one frame do I sometimes step back 5 or six
frames?
    You are viewing an MPEG with P or B frames. Because of the way these
    frames are compressed, it would take a lot of computation (ie be slow) to
    jump exactly to the frame you want, so I jump to the nearest frame that
    can be calculated at reasonable speed. If you want to know more, read the
    technical note in this package.
• Why when I try to random access a frame do I get sent to the same set of
frame numbers always?
    Same reason as above.

• Why does my MPEG have the occasional block of garbage, mostly colored
green?
    Some part of the MPEG file has become corrupt. This usually happens with
    MPEG files that have been uuencoded and have had a character or two lost
    or changed.
• Why are large parts of my MPEG filled with green garbage, or with parts
of earlier scenes.
    You have probably downloaded the MPEG file using gopher or ftp in ASCII
    format. The file is now useless. Download it again using BINARY mode.
    Better still, set your gopher or ftp application always to download .MPG
    suffix files in BINARY---don't trust its AUTOMATIC mode.
• Why is the first frame of my MPEG all green?
    Some bozos out there (probably PC users) when they create an MPEG file,
    randomly start it where they feel like it.
    The consequence is that the first some bytes of the file are garbage and
    appear as an all green first frame.
    If you have one of these MPEGs, whenever you jump to the first frame, you
    won't see the first frame, but simply the last frame you were looking at. This
    is a consequence of the way Sparkle handles errors in the MPEG file format.
    Mostly it works well, but in this particular case it isn't great.
    It is not too easy to work around this because of the many different ways in which
    the MPEG can have garbage at the start before valid data begins.

• Why at the end of an MPEG is the last frame the same as the second to
last frame?
    Some MPEGs just are created this way, with the last two frames as
    duplicates. This is a problem with those files, not a bug in Sparkle, and
    one just has to accept it.
    A similar type problem is that some video that's been converted to 30fps
    from 24fps film has duplicate frames every so often, and agin one just
    has to accept this for now.

• What does the Thread Manager do? Why do you make such a fuss over it?
    The Thread Manager provides a way for a programmer to create a number of
    tasks within an application and have those tasks all run together. So
    with Sparkle every time the user asks for something that will takes some
    time, like playing an MPEG or saving a file in some different format, I
    create a task that does that work. If the Thread Manager isn't present,
    you, the user, can't create a number of these tasks and switch between
    them---you are restricted to doing only one thing at a time.

• When I started saving a file to MPEG, it got through two frames then sat there
doing nothing for a long long time. Did it crash?
    Especially when saving to MPEG, Sparkle creates a large number of blocks
    of memory. If the mac runs out of memory partway through performing
    this saving, it will run around trying everything it can to scrounge memory.
    On my Quadra 610 I have seen it sit there for 45 seconds trying to free up
    memory before it concludes that there's nothing it can do and pops up an
    error message. I guess on slower machines it might sit there for up to two
    minutes. Just be patient and wait a while before concluding things
    have crashed.

• Will you create a PowerPC native version? A version that uses the DSP in
the AV macs?
    I don't have a PPC or an AV mac, and unless someone buys me one, I won't
    have either for sometime. Maybe when Symantec offer a Think C 7 that
    compiles to PowerPC, someone out there with a PowerPC will be able to
    do the compile for me. Until then, I'm afraid this is all that I can do.

• I tried to use the Movie Controller jog control on an MPEG and nothing
happened except that the pause/play button flickered a lot. Help.
    The jog control on the movie controller is accessed by clicking on the
    stepper buttons with the control key down.  It is used to play movies at a
    variable rate, backwards and forwards.  However since Sparkle only plays
    MPEGs at one rate (too damn slow) it doesn't do much good.  Because MPEGs
    play so slowly, I handle the timing for them differently from the way
    QuickTime times movies, and my timing doesn't mesh with what the jog control
    wants.  The flickering is there as a direct byproduct of something I added
    to give you the most useful functionality of the jog control.  If you use
    command- left and right arrows you can play the movie forwards and
    backwards, and using them one after the other will give you a very useful
    jogging capability.

• Where can I get MPEG files?
    The only site I know of is toe.cs.berkeley.edu in pub/multimedia. I don't
    have very good net access at present so I can't hunt for other sites. If
    you find other sites, tell me, and I'll add them to this list.

• Where can I find more about MPEG?
    Read the UseNet MPEG FAQ. This is published in news.answers every so
    often, and can be ftp'd as
        host: ftp.cs.tu-berlin.de
        file: /pub/msdos/windows3/graphics/mpegfaXX.zip
    (XX is a version number).
     Yeah, it's a zip file and that sucks, but that's life. There are a bunch
     of mac deZip'ers around so grab one and use it.
     There's also a version of this, I seem to remember in text format, at
         toe.cs.berkeley.edu in pub/multimedia/mpeg.
     The latest version of this file as of January 94 to be 3.0.

• How can I deal with AVI files?
    There is a program at the sumex-aim.stanford.edu ftp site, in
    info-mac/grf/util/avi-to-qt-converter.hqx that claims to do this.
    The AVI codec part of it appears to work OK, but I don't know about the
    rest of it because I have no access to AVI files. I've heard some people
    recommend it, and one guy say it crashed on his machine.

• Do I know of a program that ...?
    Not really.
    I know of no Mac program that handles MPEG sound, or MPEGs with a .WAV
    file. There is source code floating around that converts MPEG sound to
    and from 16bit 44.2kHz samples, but it does not deal with MPEG
    video/audio synching and thus isn't yet of much use.
    I know of no Mac program that can play windows AVI files.
    My next major task is handling sound.
    But until I am done with that, I'm afraid you're out of luck.

------------------------------------------------------------------------------
WHY DOESN'T SPARKLE DO...

•    Why doesn't Sparkle tell you what rate an MPEG should play at so you
    can use that rate for the QuickTime movie?
In my opinion the single stupidest thing the MPEG committee did was the
way they defined frame rates. Rather than allowing an MPEG file to be
whatever rate is wanted, like a QuickTime file, only one of eight fixed
rates is allowed. The minimum of these eight rates is 24fps. This is
clearly insane as many people don't have the hardware or the need to
display at this rate. Many MPEGs are being produced at 16 or 12fps. These
MPEGs set the frame rate to a bogus value cause they can't do anything
else. In my experience, most 120x160 MPEGs work best at about 16fps.

Now some MPEGs do give a display rate that is valid. However these are
usually large MPEGs, and their frame rate is 30fps. So unless you have a
quadra with extra hardware quicktime just won't be able to play the movie
at that rate anyway.

Because of these, for now I don't bother to point out the frame rate of
the MPEG. This will certainly change as I improve the program. Even if
you don't set that frame rate to the QT frame rate, it's interesting to
know.

Also note that some MPEGs are converted video of what was originally a
movie. Now movies run at 24fps, but video is 30 fps. So every five frames
a movie frame is doubled giving an extra video frame. This works fine
when playing at 30 fps but gives a strange jerkiness when playing at 6fps
or so. A future Sparkle enhancement will seek out these duplicate frames
and eleminate them, but for now you are stuck with them.

I know it is a pain to creat a movie at one frame rate and find it sucks,
so you have to recreate it at a second frame rate. It would be nice just
to use the movie you have but change the frame rate from say 16fps to 12
fps. This will be added to a later version of Sparkle.

•    Why doesn't Sparkle read .gl, .dl and .fli files?
One reason is that those file formats are awful. They give these dinky
low contrast horribly dithered images no-one would want to look at, and
they usually only have about 10 frames. Maybe, years from now, I'll add
those conversions, but they're about as low a priority as you get.

•    What about audio?
MPEG has audio compression as well as video compression. I do not really
understand how the audio compression works. That is, I have the MPEG
standard that explains the bit patterns used and such, but I don't
understand the overall ideas behind the encoding. I don't think I can do
a good job in terms of high speed, good quality, low memory usage and
such, unless I have such an understanding.
Does anyone out there have any references on how the MPEG psycho-acoustic
coding works? Technical journals, books, elementary or advanced
treatments. I don't care what you suggest---anything is better than my
present zero knowledge.

•    What about Video for Windows?
I would be nice to support .AVI files. But right now I know nothing about
.AVI beyond the fact that it exists. Again any info anyone has is
appreciated. Until I know how .AVI works, how it fits into the windows
environment etc, I can't even tell you if it's practical for Sparkle to
try to support .AVI, let alone start the necessary coding.

•    Why don't you write an MPEG codec?
It is not yet clear to me that an MPEG codec has any particular use.
As far as the MPEG algorithm goes, my code is not yet fast enough to give
real video, even on quadras. For a 120x160 movie I get 1 fps on my SE/30.
While I hope to get at least twice that as I continue to revise the code,
I don't think I can get much beyond it. The best I could ever get if I
wrote in pure assembly is 4 fps (the JPEG codec rate) which isn't really
good enough.
Next the MPEG file structure is rather different from the QuickTime file
structure in the way that different pieces of data are tagged and
synchronised. Things would not be like the JPEG codec where a program can
slap a short header onto a JPEG file, feed it to the JPEG codec and have
a usable picture. The program would still have to know a lot about the
MPEG file structure to use the codec, so the whole point of the codec as
being usable by programs that don't know the compression algorithm
details is lost.
Also I suspect the way MPEG uses forward image prediction in B-frames
will cause QuickTime problems both with buffer allocation (where will the
future predictive frames be stored) and in random access.

So for now I think a separate MPEG<->QT program is most useful. Of course
when Sparkle is maxed out as an application (and when QuickTime has
evolved more) I may reconsider the issue, but it won't happen soon.

•    Why does the MPEG sometimes stop before the highest frame number?
This can happen for two reasons. One is that some MPEGs (like those
produced by GC technologies) have two garbage frames at the start of the
file that cannot be decoded and are skipped by Sparkle. Right now Sparkle
isn't clever enough when it counts the nmuber of frames to notice that
these frames are garbage and should not be included in the count of total
frames. In future this smartness will be added.
A second reason is that if a frame is corrupt, it may be displayed as an
image with some junk, or it may be skipped entirely because Sparkle just
can't deal with whatever error corrupted the file.

•    Will I make the source available?
Yes, at some point. I had hoped to do so this release, but while the code
is 95% clean and readable, it's still not perfect, and don't yet think
it would help anyone much.

------------------------------------------------------------------------------
SMALL THINGS YOU MIGHT NOT HAVE NOTICED

The movie controller behaves pretty much like a standard QT controller.
You can step forward and backwards, hold down those buttons to play
forwards or backwards and click or drag in the central region to go to a
random point.
You can use the forward and backward arrow keys to step.
You can option click in the forward and backward steppers to go to the
beginning or end of the MPEG. Likewise using option forward or backward
arrow.
You can start the movie playing using either return, spacebar, or
command forward arrow. You can stop playing using spacebar or return.
You can yet play backwards by using command backward arrow.

The visual clue that the movie is being saved is that
the movie controller loses its steppers. This is not a particularly
obvious fact and may at some point be changed. For now it works once you
realize this fact.

When a movie is being saved, you can either stop the conversion
or pause it. To stop the conversion, use command-S (or the equivalent
menu option.) This will save the movie using the frames created so far.
To pause the conversion use command-P, or the equivalent menu option, or
click on the movie controller's play/pause button. The
saving will stop until you start that movie playing again. This is
occasionally useful if you want to pause a cinepak conversion to do a
short job on your mac.

If you are partway through a movie and save, the movie will be
rewound to the beginning for you. You do not need to be at the start of a
movie to save.

You can set the temporal quality options when saving to QuickTime
separately from the spatial quality options. If you need to do this, hold
down the option key while using the quality slider, and it will become a
temporal quality slider.

The code runs fastest on a 24bit screen because it then does not need to
dither the image to the display. The dithering adds a 15% or so slowdown
to the code. So if your screen has a 24bit mode, use it.

------------------------------------------------------------------------------
THE FUTURE

The basic outline for now is
    2.5 Cleaned up faster, smaller version of 2.0.
    3.0 Handles sound.
    3.5 Cleaned up faster, smaller version of 3.0.
While adding these large additions I'll fix small things as I go, and as
I have time. I don't see the user interface improving much for some
time---more important things need my time.

I also want to add stuff to allow us to create movies. Morphs, special
effects, that sort of thing.

------------------------------------------------------------------------------
HOW CAN YOU HELP IMPROVE SPARKLE?

•    Any info on psycho-acoustic encoding?
•    Any info on MicroSoft Video for Windows?
•    Assembly code for IDCT, or a way to access the JPEG codec's IDCT code.
•    Why is QuickTime so slow about creating text track movies? If you have
    code that creates text track movies fast, please show it to me.
•    Any bug reports.
•    Any ideas you have or suggestions. Your suggestions may go onto the
    list of things to do (currently two pages of single line items) but will
    probably be acted upon at some point. People have suggested several
    things to me I would not have thought of myself, so I do want your
    feedback.
•    Anything on the QT16 CD ROM beyond the technote at ftp.apple.com.
    I have full docs for QT15, but already I've learned of one aspect of 1.6
    that I could make good use of. There may be others.
------------------------------------------------------------------------------
THANKS

    Many people all over the internet have helped me write this code.
    Thanks to the people at Berkeley. Even though I'm gradually destroying
their code as I replace it with my own, they helped get this project
started.
    Thanks to the people at Stanford whose code I hope to start using real
soon now.
    Thanks to various usenet personalities who answered mac programming
questions, mailed me quicktime header files and such. Special thanks to
ldo in New Zealand, and Jon W{tte in Sweden, and bryanw, keeper of the
MPEG FAQ, who mailed me about the Stanford MPEG encoder.
    A special individual thank you goes to DS (he didn't want me to give his
name but he knows who he is.) DS mailed me a CD ROM and ten floppies of
information about QuickTime components after I complained that Apple was
not making this information easily available. Already that knowledge has
helped this version of Sparkle handle update events much better---no more
flashes of images. Once I add random access to the code, his help will
really come into its own.
    Thanks to Apple for making the greatest computers in the world.
(Though sadly they seem to be going completely clueless with regard to
how to distribute their various new ideas. I fear if they don't get their
act together about this soon, NO-ONE will support these new things
because developers will have no idea whose machine has what on it.)
    Thanks also to Symantec for creating such a great programming environment.
This program was written with Think C 5, then Think C 6 using the Think
Class Library. Having coded on Windows, X-windows, and the Mac, I can
unreservedly say that the Mac is by far the most pleasant platform for a
programmer.

Maynard Handley
[e-mail]
March 30 1994